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Introduction 


What does the United States need to do to build future generations’ knowledge and skills 
in science, technology, engineering, and mathematics (STEM), and what kinds of research and 
innovation can enable the needed changes in teaching and learning? This brief describes 
examples of research and development (R&D) sponsored by the National Science Foundation 
(NSF) that are designed to advance knowledge and practice in STEM education. It highlights just 
a few of the many R&D projects that have the potential to equip educators in pursuing national 
priorities for STEM education. 


The national priorities cited in this 
report are among those identified in recent National Priority Areas for PreK-12 STEM Education 
years by expert committees, with members Expressed in Major Reports 
drawn from the STEM professions, the 


Resources That Inspire Student Interest in STEM 


military, business and industry, as well as . Opportunities in STEM for Underrepresented 
from education policy, research, and practice. Populations 
These committees have deliberated how to . Common Core Math and Next Generation 
teach and inspire future generations of Science Standards 
STEM-skilled citizens and professionals, and ale Access to STEM Education through 
they have made specific recommendations TEC EY 

: : ; Engagement in the Practices of Science and 
for STEM education. Their recommendations EAEIneerine 
could, if carried out on a broad scale, . STEM Teacher Preparation and Professional 
transform schools and other settings where Learning 
young people learn STEM knowledge and . Learning Progressions and Formative 
skills. But for that to happen, educators need Assessment 


research-based resources and guidance that Steppe Mlechctel esses Piatt etna Ek 
they can use in changing their practice. Wrebioreiel ea cenidltiny Sus 


Some of the needed advances in R&D are likely to come from the NSF. Among the many 
kinds of work that the NSF supports are research on learning in schools and other settings and 
the development of educational innovations. Under one NSF program, Discovery Research K-12 
(DR K-12), grantees pursue research questions and develop resources, models, and tools for 
students and teachers in STEM.' 


This brief describes a sampling of DR K-12 projects that could, if successful, give life to 
the recommendations made by the experts. These are not the only such projects under way 
nationally or even within the DR K-12 program. They were selected to illustrate the wide range 
of ways in which the education R&D community, working with STEM professionals and STEM 
educators, is studying and developing promising ideas that may advance teaching and learning. 


For this brief, staff of the Community Advancing Discovery Research in Education 
(CADRE)’ reviewed recent national reports to identify recommendations for STEM education 


' Additional information about the DR K-12 program and its projects can be found on the NSF’s DR K-12 website. 
CADRE, funded by the National Science Foundation, is a resource network that connects STEM education 
researchers and developers in the DR K-12 program community. Additional information about CADRE and DR K- 
12 projects can be found on the CADRE website at www.cadrek12.org. 
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that will depend on better resources, models, and tools for their enactment. The reports were 
issued by bodies convened by the National Academy of Sciences, U.S. Department of Education, 
President’s Council of Advisors on Science and Technology, National Science and Technology 
Council, Council on Foreign Relations, and Gordon Commission on the Future of Assessment in 
Education (see the Appendix for a full listing of the reports). CADRE identified nine priorities 
that recur across multiple reports, each having direct implications for practice in teaching and 
learning. 


For each priority, this brief presents several specific recommendations culled from the 
reports, followed by capsule descriptions of a few of the DR K-12 projects that are already 
working in these areas.’ These projects 
Discovery Research K-12 Program serve as examples, intended to illustrate the 
rich variety of work under way in the DR 
The National Science Foundation’s Discovery Research K-12 portfolio. The ones highlighted here 
K-12 (DR K-12) program arid) its oe projects seek to were chosen because they differ in their 
enhance learning and teaching in PreK-12 STEM : ere 

i : intended outcomes, discipline areas, 
education. DR K-12 projects focus on current and future : 
needs in STEM education by researching and populations served, methods, and scope. 
developing innovative resources, models, and tools. The What they have in common is that each one, 
program seeks to identify and develop the next if successful, could ultimately support 
generation of STEM education innovations to improve schools and other learning settings in 
student learning in schools and other settings. bringing about the changes in STEM 
education that experts say the nation needs. 


1 Resources That Inspire Student Interest in STEM 


Several major national reports call for an increased focus on student engagement, 
motivation, and interest in the STEM fields. In many of today’s classrooms, content and 
pedagogy are based on reading texts and memorizing facts, doing little to excite and challenge 
young minds. Students have too few opportunities to conduct hands-on experiments and connect 
concepts taught in the classroom to their daily lives. Instruction in STEM could do more to ignite 
their curiosities and passions, say these reports. That being said, there are promising efforts in 
this direction, including those that have students conduct real world and simulated experiments, 
engage in mathematical and scientific inquiry, solve authentic design problems, and learn about 
scientific issues of the day. Recommendations in the reports encourage the R&D field to 
embrace student interest in STEM as a goal and to harness the affordances of technology and 
experiences outside the traditional classroom setting. 


“Increase interest and engagement in STEM among children and adults, especially those from 
groups traditionally underrepresented in STEM, so that learners are motivated to explore and 
participate in STEM throughout their lives.” — Coordinating Federal in Science, Technology, 
Engineering, and Mathematics (STEM) Education Investments: Progress Report, 2012 


> DR K-12 project descriptions in this brief were developed by CADRE staff using publicly available information, 
primarily project abstracts housed on the NSF website and the projects’ own websites. Project leaders did not 
provide input on the content of the descriptions. 
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“Conduct research and development that explores how embedded assessment technologies, such 
as simulations, collaboration environments, virtual worlds, games, and cognitive tutors, can be 
used to engage and motivate learners while assessing complex skills.” — National Education 
Technology Plan, 2010 


“Create opportunities for inspiration through individual and group experiences outside of the 
classroom.” — Prepare and Inspire: K-12 Education in Science, Technology, Engineering, and Math 
(STEM) for America’s Future, 2010 


Consistent with these recommendations, the DR K-12 community conducts R&D on 
resources that show promise for engaging students in STEM learning and, in turn, increasing 
student interest in pursuing STEM degrees and careers. For instance, DR K-12 projects 
investigate the capacity of online games, social media devices, virtual tutors, problem-based 
curricula, and real-world immersion experiences to excite students of all ages and backgrounds, 
especially those from traditionally underserved backgrounds. 


Engaging Youth through Engineering Module Study — Susan Pruet, Mobile Area Education Foundation (MAEF) 


Engaging Youth through Engineering (EYE) is a workforce development initiative with the goal of inspiring and 
preparing middle school students to become skilled technology-savvy workers in Mobile, Alabama industries, such 
as aerospace and shipbuilding. EYE has developed a series of week-long engineering units that challenge students 
to solve real-world engineering design problems, such as determining how to reduce the rate of sediment runoff 
into Mobile Bay. Teachers receive training on the EYE modules and on how to help students use the engineering 
design process to apply their math and science knowledge. Volunteer STEM professionals bring their passion and 
expertise to EYE activities. Co-Principal Investigators (Co-Pls): James Van Haneghan and Robert Foley, both of 
University of South Alabama; Martha Peek, Mobile County Public Schools. Abstract. Website. 


Enhancing Games with Assessment and Metacognitive Emphases (EGAME) — Douglas Clark, Vanderbilt University 


EGAME teaches physics to middle-school students through use of digital games. The game-based environment 
tailors content for each student’s needs to enhance his or her learning of Newtonian mechanics and other physics 
concepts. Players predict, reflect, solve puzzles, and explain their thinking, all while mounting rescue missions. 
Using computer adaptive testing, the game aims to diagnose the players’ understanding and engage them in 
increasingly complex content. To advance the theory and design of game-based learning environments, the project 
includes a randomized experimental research design. CO-Pls: Jim Minstrell, FACET Innovations; Daniel White, 
Filament Games; Gautam Biswas and Pratim Sengupta, both of Vanderbilt University. Abstract. Website. 


Expanding PhET Interactive Science Simulations to Grades 4-8: A Research-based Approach — Katherine Perkins, 
University of Colorado at Boulder 


PhET is a celebrated source for fun, free, interactive, computer simulations of physical phenomenon. Through 
engagement with the simulations, students are able to make connections between the underlying science in the 
simulations and real-life phenomena in the physical world. Researchers in this project are studying 25 existing 
science simulations to learn how various characteristics of simulation design influence student engagement and 
learning, with the goal of identifying successful design features. PhET will then formulate generalized design 
guidelines, which will be tested through the development of 10 new simulations. Co-P/s: Michael Dubson and Noah 
Podolefsky, both of University of Colorado at Boulder; Daniel Schwartz, Stanford University. Abstract. Website. 
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ScratchJr: Computer Programming in Early Childhood Education as a Pathway to Academic Readiness and 
Success — Marina Bers, Tufts University; Mitchel Resnick, Massachusetts Institute of Technology 


Scratch, developed with NSF support, is a widely implemented computer programming language that allows youth 
to program their own games, stories, and animations and to share them around the world. A newer version of 
Scratch, ScratchJr, is now being designed and tested specifically for early childhood education (K-2). ScratchJr will 
enable the youngest students to learn basic programming, think creatively, reason, communicate, and work 
collaboratively. Students will be able to study foundational academic concepts and skills while toying around with 
ScratchJr for free at home and at school. Abstract. Website. 


Multiple Instrumental Case Studies of Inclusive STEM-focused High Schools: Opportunity Structures for 
Preparation and Inspiration (OSPrl) — Sharon Lynch, George Washington University 


OSPrl is an investigation into the emerging trend of inclusive STEM-focused high schools. The study will learn the 
ways these schools help students develop an interest in STEM and pursue STEM careers. For instance, it will look at 
opportunities for authentic STEM engagement through mentorships with STEM professionals, internships at local 
businesses, and early college enrollment in STEM courses. Focusing on 12 case study high schools, researchers will 
analyze approaches to coursework development, instructional strategies, and workforce development. It will also 
inform the field on student STEM trajectories from these schools, including student graduation and college 
admission rates. Co-Pls: Barbara Means, SRI; Erin Peters Burton, George Mason University; Tara Behrend, George 
Washington University. Abstract. Website. 


2 Opportunities in STEM for Underrepresented Populations 


Historically, students from ethnic minority backgrounds, female students, and students 
with disabilities have not pursued STEM degree in equal proportions to their Caucasian and 
Asian male peers. The education system may be leaving untapped a large pool of talented STEM 
professionals from these groups. Several national reports advocate for an assessment of current 
educational practices and the development of new PreK-12 strategies to interest students from 
underrepresented populations in STEM fields. The reports call for efforts to better understand 
their barriers to participation and effective methods to engage these students in STEM content 
and career pathways. Developing STEM talent among underserved or underrepresented groups 
not only benefits the individuals entering STEM professions, but also likely contributes to the 
strength of those professions and to America’s global competitiveness. 


“Increase America’s talent pool by vastly improving K-12 mathematics and science education for 
underrepresented minorities.” — Expanding Underrepresented Minority Participation: America’s 
Science and Technology Talent at the Crossroads, 2011 


“Increase the number of individuals from underrepresented groups that graduate with STEM 
degrees.” 
— Coordinating Federal STEM Education Investments: Progress Report, 2012 


“Expand the number of students who ultimately pursue advanced degrees and careers in STEM 
fields and broaden the participation of women and minorities in these fields.” — Successful K-12 
STEM Education: Identifying Effective Approaches in Science, Technology, Engineering, and 
Mathematics, 2011 
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In the DRK12 program, projects develop and investigate ways to improve STEM 
education opportunities for underrepresented youth. For example, some examine ways in which 
to better prepare teachers to engage underrepresented groups or to expose those students to 
STEM content in a way that is culturally and linguistically relevant. In many projects, curriculum 
and teacher professional development is strengthened or developed to meet the unique needs of 
different groups of learners, often using technological innovations, multimedia platforms, and/or 
immersion in real-world settings that generate mastery of skills and interest in STEM careers. 


Community-Based Engineering Design Challenges for Adolescent English Learners — Amy Wilson, Utah State 
University 


In this ethnographic study, adolescent English language learners will solve engineering design problems using 
resources in their community. Latino students work collaboratively to identify an engineering design need in their 
community and design a solution, drawing on online and offline resources, including those that are social, cultural, 
and linguistic. The study will produce frameworks and inform the future development of curricular materials and 
professional development models for culturally diverse students. Co-Pls: Christine Hailey and Daniel Householder, 
both of Utah State University. Abstract. 


Supporting Computational Algorithmic Thinking (SCAT) - Exploring the Development of Computational 
Algorithmic Thinking Capabilities in African-American Middle School Girls — Jakita Thomas, Spelman College 


SCAT seeks to develop the computational thinking and problem solving skills of African-American middle-school 
girls. Over a three-year period, the girls attend two-week summer camps in which they learn to design interactive 
games that require the use of math, programming, and reasoning. The middle-schoolers work with programming 
experts and undergraduate computer science majors from Spelman College, who also help the girls explore STEM 
career pathways. SCAT will develop lesson plans and activity materials for free public use. Abstract. Website. 


Morehouse College DR K-12 Pre-Service STEM Teacher Initiative — Cynthia Trawick, Morehouse College 


Eighty promising eleventh-grade African-American male students from the Atlanta Public School system will be 
selected to participate in a STEM teacher preparation program. The six week intensive summer program, which 
includes twelfth grade Saturday Academies, will integrate STEM education with teacher preparation and 
mentoring to develop student interest in becoming secondary teachers with STEM discipline expertise. The goal of 
the program is to recruit and prepare African-American males for careers in secondary math and science teaching. 
The program will identify successful strategies and challenges for the recruitment and retention of African 
American males in STEM teaching careers. Co-Plis: Charles Meadows and Abdelkrim Brania, both of Morehouse 
College. Abstract. Website. 


The Role of Educative Curriculum Materials in Supporting Science Teaching Practices with English Language 
Learners — Jacqueline Barber, University of California-Berkeley 


A team of curriculum developers and researchers from various disciplines will identify ways in which the teaching 
of STEM curricula can be adapted to meet the needs of English language learners. The study will investigate the 
NSF-funded Seeds of Science/Roots of Reading program to determine whether curriculum materials that are 
designed to support teacher learning, as well as student learning, can support teachers in meeting the instructional 
needs of English language learners. This effort will gather evidence to set a new standard for curriculum materials 
that help teachers effectively meet the needs of an increasing linguistically diverse student body. Co-Pis: Jonna 
Kulikowich, Pennsylvania State University; Marco Bravo, Santa Clara University; Gina Cervetti, University of 
Michigan. Abstract. 
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Nurturing Multiplicative Reasoning in Students with Learning Disabilities in a Computerized Conceptual- 
Modeling Environment (NMRSD-CCME) — Yan Ping Xin, Purdue University 


NMRSD-CCME will develop a computer system designed to help students with learning disabilities master 
multiplicative reasoning. The program will meet individual students’ needs by allowing students to work 
independently on problems adapted to their level of understanding and by providing them with recommendations 
of tasks in order to advance to the next level. Teachers will be provided with protocols to implement the computer 
system in the classroom and will have a better tool with which to assess student mastery of multiplicative 
reasoning skills. The ultimate objective is to develop a computer-based learning system that improves math 
achievement and confidence among students with disabilities. Co-Pls: Ron Tzur, University of Colorado - Denver; 
Luo Si, Purdue University. Abstract. 


3 Common Core Math and Next Generation Science 
Standards 


To date, 45 states, four territories, the District of Columbia, and the Department of 
Defense Education Activity have adopted the Common Core State Standards (CCSS),* a set of 
standards in mathematics and English language arts created by a consortium of states to establish 
a common understanding nationwide of what students at each grade level should be expected to 
know. Subsequently, a consortium of states developed the Next Generation Science Standards 
(NGSS),° now being adopted by states, to establish a common set of standards for science. 
However, the success of these math and science standards in moving students forward depends 
on the implementation of the standards in the classroom. Several national reports on STEM 
education recommend an investment in and commitment to the national standards, and they call 
for the R&D community to embark on the development and research of resources that can 
contribute to effective classroom instruction. Report recommendations highlight the need for 
professional development on standards implementation, as well as for aligned curricular and 
assessment resources. 


“The Federal Government should vigorously support the state-led effort to develop common 
standards in STEM subjects, by providing financial and technical support to states for (i) 
rigorous, high-quality professional development aligned with shared standards, and (ii) the 
development, evaluation, administration, and ongoing improvement of assessments aligned to 
those standards.” — Prepare and Inspire: K-12 Education in Science, Technology, Engineering 
and Math (STEM) for America’s Future, 2010 


“Districts should ensure...4. Adoption of instructional materials in grades K-12 that embody 
Common Core State Standards in mathematics and A Framework for K-12 Science Education and 
5. Classroom coverage of content and practices in Common Core State Standards in 
mathematics and A Framework for K-12 Science Education.” — Successful K-12 STEM Education: 
Identifying Effective Approaches in Science, Technology, Engineering, and Mathematics, 2011 


* More information regarding the CCSS-Mathematics can be found at www.corestandards.org. 
° More information regarding the NGSS can be found at www.nextgenscience.org. 
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“Governors should work with educators, leaders in industry, the military, and others to expand 
the Common Core State Standards... State policymakers should work with educators and a 
broad coalition of partners to ensure that new exams aligned with the Common Core are more 
meaningful assessments of student learning.” — U.S. Education Reform and National Security 
(Independent Task Force Report No. 68), 2012 


Many DR K-12 projects are conducting work that is aligned with the CCSS and NGSS, 
and several of them are explicitly focused on the development and testing of high-quality 
materials that lead to effective implementation of the new national standards. With the CCSS 
mathematics standards requiring a change in instructional practice, some projects are developing 
aligned resources to help teachers improve their skills. Others are developing frameworks that 
detail how learners progress step-by-step toward goals in the standards. These frameworks can 
inform the development of other curricula and assessments. DR K-12 projects are also 
developing educational games and curricula focused on improving engineering knowledge and 
skills, an area of NGSS focus. 


Implementing the Mathematical Practice Standards (IMPS): Enhancing Teachers' Ability to Support the Common 
Core State Standards — E. Paul Goldenberg, EDC 


IMPS will produce a set of professional development materials to help math teachers implement the CCSS. 
Materials will include professionally produced videos and print materials that show specific, high-quality ways of 
teaching mathematical practices described in the standards. An evaluative component will examine what features 
of the materials teachers find most useful, as well as which types of professional development activities help 
teachers implement the teaching practices. Co-P/s: Mark Driscoll, Albert Cuoco, June Mark, and Deborah Spencer, 
all of EDC. Abstract. Website. 


Developing Principles for Mathematics Curriculum Design and Use in the Common Core Era — Jeffrey Choppin, 
University of Rochester 


Researchers at a consortium of universities are developing principles for the design and use of curriculum 
resources aligned with the CCSS-Mathematics. The principles are informed by data collection on the efficacy of 
particular elements of currently available curricula used by teachers nationwide for students in grades 6-8. 
Outcomes of the project include guidelines for curriculum developers and school leaders in making resources 
relevant, accessible, and easily translated into instruction. Co-Pls: Corey Drake, Michigan State University; Amy 
Roth McDuffie, Washington State University Tri-Cities; Jon Davis, Western Michigan University. Abstract. 


Completing, Validating, and Linking Learning Trajectories for K-8 Rational Number Reasoning Tied to the 
Common Core Standards — Alan Maloney, North Carolina State University 


Investigators are developing learning trajectories in three areas of mathematics and accompanying assessment 
items for students in grades 3-7. The learning trajectories and assessment items are aligned with the CCSS- 
Mathematics and will be field-tested and validated through a series of observations, interviews, and assessments 
of students. The outcome of the project is a set of electronically available supporting materials for teachers that 
will help them help them successfully implement the CCSSM. The learning trajectories can also inform the 
development of other curricula and assessments. Abstract. Website. 
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An Examination of Science and Technology Teachers' Conceptual Learning through Concept-Based Engineering 
Professional Development — Rodney Custer, Black Hills State University 


With an eye toward the NGSS, this project will infuse engineering into life and physical sciences and develop 
supportive professional development materials. Secondary school science teachers attend two weeks of 
summertime professional development, in which they deepen their knowledge of engineering concepts and learn 
how to develop engineering-infused science modules for use in their classrooms during the school year, then 
debrief and revise their work the following summer. Products will include practical resources for secondary science 
teachers to use in implementing lessons aligned with the NGSS. Co-Pls: Julia Ross, University of Maryland — 
Baltimore County; Jenny Daugherty, Purdue University. Abstract. 


panded Access to STEM Learninc h Technolog 


While great progress has been made in using the power of technology for educational 
purposes, some national reports suggest there should be a continued focus on harnessing 
technology to make STEM educational resources more accessible. Online resources can open up 
learning opportunities for students in locations or at times outside of traditional classroom 
instruction. Technological innovations can connect learners with one another and with teachers 
in ways that were not previously available. Technology puts an unprecedented amount of 
educational resources at the fingertips of students and teachers, and it can enable resources to be 
organized in ways that allow individual learners to pursue their own STEM learning goals. 
Lastly, technology makes it feasible to establish platforms in which users and user communities 
can generate, share, adapt, and curate free STEM learning resources, further expanding and 
repurposing what is available. 


“States, districts, and others should develop and implement learning resources that exploit the 
flexibility and power of technology to reach all learners anytime and anywhere.” — National 
Education Technology Plan, 2010 


“Use technology to provide all learners with online access to effective teaching and better 
learning opportunities and options especially in places where they are not otherwise available.” 
— National Education Technology Plan, 2010 


“R&D funding should promote the development and sharing of open educational resources that 
include assessment items that address learning transfer.” — Expanding Evidence Approaches for 
Learning in a Digital World, 2013 


The DR K-12 portfolio includes a multitude of projects pursuing technological 
innovations, many of which are intended to increase the availability of STEM education 
resources. This includes 154 projects (44 percent of the DR K-12 portfolio as of 2012)° that are 
developing computer or internet activities and resources for student STEM learning. Some 
project products operate on mobile devices to enable portable learning and field research. Other 
projects are developing collaborative online educational games, courses, or community learning 
networks, while still others are exploring approaches to online intelligent tutoring that can occur 
anytime and target individual needs. 


° CADRE. (2013). Descriptive overview of the Discovery Research K-12 (DR K-12) portfolio: Projects funded from 
2007 to 2012, p.11. 
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CyberSTEM: Making Discovery Visible Through Digital Games — Richard Halverson, University of Wisconsin- 
Madison 


CyberSTEM connects homes, schools, and other informal learning settings, such as museums, to an integrated 
gaming platform where learners in school and in other locations can come together to study cutting-edge 
discoveries in life sciences. CyberSTEM will develop five science-based games with an associated curriculum that 
are designed to increase engagement and learning in science. Targeting students in grades 6-9, the program will 
use informal gaming channels, such as Kongregate, iTunes, and Xbox Live, to reach the general public who will 
ultimately create the community that sustains CyberSTEM. Co-Pls: Susan Millar and Kurt Squire, both of University 
of Wisconsin-Madison. Abstract. 


Arcadia: The Next Generation - Transforming STEM Learning Through Transmedia Games — Jodi Asbell-Clarke, 
TERC, Inc. 


Students participating in the Arcadia project use smartphones for scientific investigation. The project examines 
how social media gaming can support high-quality scientific inquiry across a diverse community of learners. 
Designers and researchers are integrating web-based social media and smartphone applications into an 
experimental environment that promotes scientific investigation and STEM learning. They intend to inform future 
learning games for use inside and outside of schools by a diverse population of students and families. Co-Pls: Jamie 
Larsen and Teon Edwards, both of TERC, Inc. Abstract. Website. 


The Leonardo Project: An Intelligent Cyberlearning System for Interactive Scientific Modeling in Elementary 
Science Education — James Lester, North Carolina State University 


The Leonardo Project studies ways in which children in grades 4-5 learn through the use of technology. Using 
intelligent virtual science notebooks, students create multimedia artifacts that combine animation, sound, and 
narration in experiments with models of physical phenomena. Investigators then identify and study the 
characteristics of the technologies and conditions that most successfully help students master scientific concepts. 
Co-PIS: Bradford Mott, Michael Carter, and Eric Wiebe, all of North Carolina State University. Abstract. Website. 


Researching the Expansion of K-5 Mathematics Specialist Program in Rural School Systems — Reuben Farley, 
Virginia Commonwealth University 


Using a combination of distance and face-to-face learning, remote mathematics specialists will develop new skills 
for improving student mathematics achievement in rural schools. R&D in this project will contribute knowledge to 
the field regarding the effectiveness of distance education for STEM educators. In the process, students and 
teachers who previously had limited access to high-quality mathematics resources will receive coaching support 
enabled through technology. The project will assess the model’s impact on student outcomes as defined by state 
standards. Co-Pls: Vickie Inge, University of Virginia; Patricia Campbell, University of Maryland; Aimee Ellington and 
Joy Whitenack, both of Virginia Commonwealth University. Abstract. Website. 


5 Engagement in the Practices of Science and Engineering 


Several national reports made a case for teaching science and engineering as practices— 
ones carried out by professionals and students of all ages, and done so as to answer real questions 
and solve real problems. While not disregarding the importance of scientific content knowledge, 
these reports advocate for STEM education that engages students in actually “doing” science and 
engineering. A single-minded focus on developing content knowledge, absent scientific inquiry, 
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can give students a narrow impression of science as only a collection of facts with little use. 
Instead, the reports advance the notion that preK-12 STEM education should expose students to 
the ways in which scientific knowledge is developed and involve them in authentic practices 
used by scientists. They likewise promote the teaching of engineering as a process for designing 
solutions through the practical application of science and math content knowledge. 


“Developers of curricula and standards should present science as a process of building theories 
and models using evidence, checking them for internal consistency and coherence, and testing 
them empirically. Discussions of scientific methodology should be introduced in the context of 
pursuing specific questions and issues rather than as templates.” — Taking Science to School: 
Learning and Teaching Science in Grades K-8, 2007 


“Development of curricular and instructional materials...[should be] focused on scientific and 
engineering practices, crosscutting concepts, and disciplinary core ideas.” — A Framework for K- 
12 Science Education: Practices, Crosscutting Concepts, and Core Ideas, 2012 


“States and national organizations should develop effective systems of assessment aligned with 
the next generation of science standards and that emphasize science practices rather than mere 
factual recall.” — Successful K-12 STEM Education: Identifying Effective Approaches in Science, 
Technology, Engineering, and Mathematics, 2011 


Consistent with these recommendations, many developers in the DR K-12 community 
root their materials in the practice of science and engineering, while also imparting knowledge of 
specific content. As of 2012, 41 percent of projects in the program’s portfolio focused on the use 
of scientific inquiry procedures.’ Project resources seek to teach scientific and engineering 
methods through active engagement in those methods, with students designing investigations, 
conducting experiments, solving engineering design challenges, and presenting their findings. 
DR K-12 projects expose students to the real practice of science and engineering that is driven by 
the human desire to answer questions and solve problems. 


Evaluation of the Sustainability and Effectiveness of Inquiry-Based Advanced Placement Science Courses: 
Evidence from an In-Depth Formative Evaluation & Randomized Controlled Study — Mark Long, University of 
Washington 


Many high school students across the country enroll in Advanced Placement (AP) Biology and Chemistry courses. 
This project will determine course impact on students’ scientific inquiry skills, depth of disciplinary knowledge, and 


enrollment in college STEM majors. This randomized control trial study answers whether AP courses produce 
students with authentic scientific skills that continue on the path toward becoming scientists and engineers. It will 
provide feedback on how to improve the courses and related teacher training, and it will inform schools and 
policymakers on challenges as enrollment in the AP science courses expands. Co-Pls: Raymond McGhee, SRI; Dylan 
Conger, George Washington University. Abstract. 


7 CADRE (2013), p. 8. 
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Science Learning: Integrating Design, Engineering and Robotics (SLIDER) — Richard Millman, Georgia Tech 
Research Corporation 


Partnering with a state department of education and three districts, the SLIDER project is developing a physical 
science curriculum that engages students in problem-based inquiry and engineering design. Students are 
presented with challenges that require them to design and evaluate products that solve specific problems. SLIDER 
integrates engineering and robotics to teach core concepts in mechanics, waves, and energy, while increasing 
student motivation, creativity, and interest in STEM fields. Co-Pls: Juan-Carlos Aguilar, Georgia Department of 
Education; Donna Llewellyn and Marion Usselman, both of Georgia Institute of Technology. Abstract. Website. 


SAVE Science: Situated Assessment Using Virtual Environments for Science Content and Inquiry — Diane Jass 
Ketelhut, University of Maryland 


In SAVE Science, students engage in collaborative games to carry out authentic scientific practices, and teachers 
receive assessment data they can use to quickly adjust instruction for individual needs. This project is creating an 
innovative virtual system to evaluate the learning of science inquiry and core concepts. Rather than assessing 
student learning through rote answers, this project embeds scientific activities and assessments into virtual 
environments. Co-Pls: Brian Nelson, Arizona State University; Catherine Schifter, Temple University. Abstract. 
Website. 


Ready for Robotics: The Missing T and E of STEM in Early Childhood Education — Marina Bers, Tufts University 


Ready for Robotics enlists young children as engineers when they play with gears, levers, motors, and sensors, with 
robotics activities acting as a gateway to applied mathematics and problem solving. The project team is designing a 
low-cost robotics construction kit for use in early childhood classrooms (PrK-2), as well as a teacher professional 
development model on engineering and technology instruction for young children. Abstract. Website. 


The Evidence Games: Collaborative Games Engaging Middle School Students in the Evaluation of Scientific 
Evidence — Janis Bulgren, University of Kansas 


The Evidence Games are a series of interactive on-line games designed to increase middle schoolers’ skills in 
scientific argumentation and ability to evaluate scientific evidence. The games, which increase in their complexity, 
begin with simple claims and evidence and advance to the analysis of claims that are current in the field of science. 
Students can also challenge each other to evaluate their own claims and evidence, further developing the critical 
thinking and communication skills needed by scientists. Co-Pls: James Ellis, Bruce Frey, and Marilyn Ault, all of 
University of Kansas. Abstract. 


6 STEM Teacher Preparation and Professional Learnine 


In order for students to make gains in STEM subjects, their teachers must first be 
knowledgeable about STEM content, understand how students learn science, and be able to 
choose effective tools to deliver STEM instruction to their students. National reports call for 
improved opportunities for teachers, including at the elementary levels, to improve their own 
STEM education skills and knowledge. R&D efforts should determine the qualities of effective 
professional development in STEM education, as well as barriers to implementation. Just as 
further research is needed to discover how students learn STEM content and skills, the same is 
true for understanding how STEM teachers become excellent educators. The reports also push 
for teachers to have on-demand access to the resources and learning opportunities that they need 


© 2013 CADRE 11 


to do their job well, including those that are job-embedded or enhanced through technology. The 
need for improved STEM teacher preparation and professional learning activities is one of the 
most frequently mentioned recommendations in national reports focused on K-12 STEM 
education. 


“There is emerging consensus that high-quality professional development (a) focuses on 
developing teachers’ capabilities and knowledge to teach content and subject matter, (b) 
addresses teachers’ classroom work and the problems they encounter in their school settings, 
and (c) provides multiple and sustained opportunities for teacher learning over a substantial 
time interval...additional research is needed to determine the characteristics of effective 
professional development for science and mathematics teachers and potential obstacles to 
engaging in professional development.” — Monitoring Progress Toward Successful K-12 STEM 
Education: A Nation Advancing ?, 2012 


“State and local school systems should ensure that all K-8 teachers experience sustained 
science-specific professional development in preparation and while in service. Professional 
development should be rooted in the science that teachers teach and should include 
opportunities to learn about science, about current research on how children learn science, and 
about how to teach science.” — Taking Science to School: Learning and Teaching Science in 
Grades K-8, 2007 


“Supporting teachers’ learning:...Continued research is needed to better understand the 
possible longitudinal trajectories that K-12 teachers may take in becoming knowledgeable and 
accomplished science teachers.” — A Framework for K-12 Science Education: Practices, 
Crosscutting Concepts, and Core Ideas, 2012 


“Expand opportunities for educators to have access to technology-based content, resources, 
and tools where and when they need them.” — National Education Technology Plan, 2010 


Below is a sampling of the 235 projects (67 percent of the DR K-12 portfolio as of 2012)° 
that include the development or assessment of resources for STEM teacher professional 
development. The DR K-12 program includes a wide range of delivery approaches, including 
intensive job-embedded coaching, immersive workshops, tools for practice, online self-guided 
modules, virtual professional learning communities, and residency programs. Together, these 
projects deliver professional development to teachers across the spectrum of educational settings 
in the United States, from teachers in urban, diverse districts to teachers in the rural Midwest. 
The results of these projects are intended to contribute to the field of knowledge about what 
components of professional development and delivery methods are most effective in furthering 
the content knowledge and pedagogical prowess of teachers of STEM subjects. 


* CADRE (2013), p. 11. 
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Teacher Residency Academy Alliance — William McHenry, Jackson State University 


Teacher residency programs are gaining attention as a promising tool for teacher preparation and development, 
yet there is a need for addition research on their merit. The Teacher Residency Academy Alliance (TRA2) develops 
and studies a teacher residency academy for training middle school and secondary science teachers who will teach 
in high-needs, diverse schools in seven districts in Mississippi and Louisiana. The goals of the project are to 
increase the number of highly effective science educators from underrepresented minority populations and to 
conduct research on the implementation of a clinically-based preparation program. Co-Pls: Rosalind Hale, Xavier 
University of Louisiana; Johnnie Mills-Jones, Jackson State University; Mary Dilworth, National Board of 
Professional Teaching Standards. Abstract. 


Teachers Helping Teachers Teach Science Inquiry: The "Just ASK" Project — James Shymansky, University of 
Missouri at St. Louis 


The “Just ASK” project establishes online communities of practice for K-6 teachers in nine rural elementary schools 
in lowa and Missouri. The online platform enables teacher-to-teacher video and audio connections and the 
exchange of lesson plans or other instructional resources, as well as connecting teachers with professional 
development specialists. Adapting Science Kit lessons and videos are posted on the site to be accessed at any time. 
Researchers will investigate the impact of the communities of practice on teacher pedagogy and student learning, 
and use the data to inform the expansion of the project in the future. Abstract. Website. 


Tool Systems to Support Progress toward Expert-Like Teaching by Early Career Science Educators — Mark 
Windschitl, University of Washington 


High-leverage expert-like teaching practices have the potential to change teaching and student outcomes. This 
project has developed a web-based system intended to help early career and pre-service teachers move more 
quickly along the path toward becoming expert teachers. The system includes practical tools to help teachers 
develop inquiry-based science lessons, scaffold inquiry conversation in classrooms with diverse student 
populations, and analyze student data to inform tailored instruction to individual students. Researchers are 
studying the efficacy of the tools and the system in helping teachers improve their teaching and student ability to 
provide evidence-based arguments during classroom discussion. Co-P/: Jessica Thompson, University of 
Washington. Abstract. Website. 


Energy: A Multidisciplinary Approach for Teachers (EMAT) Designing and Studying a Multidisciplinary, Online 
Course for High School Teachers — Susan Kowalski, Biological Sciences Curriculum Study 


EMAT is an online professional development course to train high school science teachers, particularly those 
teaching outside of their endorsement area and working with students underrepresented in science professions. 
The course, which focuses on energy-related concepts, has teachers reflect on research-based teaching practices. 
This project will provide new knowledge on the elements of effective, online professional development for 
teachers by collecting longitudinal data on teachers’ science and pedagogical content knowledge and the impact 
on student learning. Co-Pls: Kimberly Obbink, Montana State University; Pamela Van Scotter, Biological Sciences 
Curriculum Study; David Davis, Oregon Public Broadcasting. Abstract. 
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Mathematics for Teaching: A Problem-based Resource for Teachers — Phillip Griffiths, Institute for Advanced 
Study 


Secondary mathematics educators and mathematicians are working together to design a secondary mathematics 
curriculum and professional development courses. This approach capitalizes on the strengths of the two groups, as 
mathematicians ensure curriculum rigor and quality and the mathematics educators make the curriculum relevant 
and accessible for practicing secondary math teachers. The mathematics educators are trained as facilitators for 
the professional development, which they deliver to practicing secondary mathematics educators throughout the 
country. Co-Pls: Albert Cuoco, EDC; C. Clemens, Institute for Advanced Study. Abstract. 


7 Learning Progressions and Formative Assessment 


Together, learning progressions and formative assessment enable one to identify gaps or 
needs in the learner’s knowledge, and then apply a developmentally appropriate intervention that 
offers the best chance at progress toward more complex understandings. While R&D work on 
learning progressions and formative assessment are sometimes conducted separately, they are 
tightly connected in that developers and practitioners of formative assessment often rely on 
learning progressions to determine the next step in instruction—the step that will effectively 
move the learner closer towards the learning goals. Below, this report first discusses learning 
progressions, then moves on to address formative assessment. 


Learning progressions, and their variation learning trajectories, focus on an explicit 
concept and seek to specify the sequence of building blocks or steps that lead to mastery of the 
concept. In the DR K-12 program and elsewhere, there is a commitment to the rigorous testing 
and refinement of learning progressions in order to have confidence that they accurately express 
the path to a goal. Developing learning progressions is often done incrementally, building on 
prior work to develop new progressions for different grades or for related concepts. They are 
more specific and research-based than a typical curriculum’s scope and sequence. Ultimately, 
their purpose is to inform the development of other educational resources, primarily assessments, 
curricula, models of instructional practice, and professional development. 


“Learning progressions in science and engineering: ...Research may focus on a particular core 
idea and ask what sequence of learning experiences, including engagement in practices, around 
that idea best advance student understanding and address common misconceptions.” — A 
Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas, 2012 


“Designers and developers of instruction targeted at deeper learning and development of 
transferable 21" century competencies should begin with clearly delineated learning goals and a 
model of how learning is expected to develop, along with assessments to measure student 
progress toward and attainment of the goals.” — Education for Life and Work: Developing 
Transferable Knowledge and Skills in the 21° Century, 2012 


The DR K-12 program, which focuses on the development of STEM education resources, 


models, and technologies, includes 42 projects (12 percent of the DR K-12 portfolio as of 2012)° 
that develop learning progressions for specific concepts and grade spans. Projects extend the 


° CADRE (2013), p. 13. 
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knowledge developed in others, and many develop resources in concert with the development 
and testing of a related learning progression. 


A Learning Progression-based System for Promoting Understanding of Carbon-Transforming Processes (CCE) — 
Charles Anderson, Michigan State University 


A collaborative effort, this project is developing a framework that details the progression students undergo as they 
learn increasingly complex principles related to carbon-transforming processes (e.g., metabolism, combustion, 
carbon sequestration). It will produce a suite of materials for students and teachers in middle and high school, 
including a validated learning progression, tools for student reasoning, twelve curricular units, a set of teaching 
strategies, refined assessments, and materials for professional development. The framework can inform the work 
of other curriculum and assessment developers. Co-Pls: Mark Wilson, University of California at Berkeley; Kathleen 
Schwille, National Geographic Education; Lindsey Mohan, Michigan State University; Daniel Gallagher, Bellevue 
School District and Seattle Public Schools. Abstract. Website. 


Learning Trajectories to Support the Growth of Measurement Knowledge: Pre-K through Middle School — Jeffrey 
Barrett, Illinois State University 


Measurement, a domain of knowledge included in most academic standards including the Common Core, is 
addressed in math classrooms across the grade span and throughout the country. This project seeks to develop 
and test the sequence in which individuals learn geometric measurement for the ten-year span of human 
development, PreK to Grade 8. The learning trajectories produced will provide a state-of-the-art interdisciplinary 
model of how a student’s understanding of measurement builds, at which point they may inform the subsequent 
development of standards-focused curricula and assessment. Co-Pls: Craig Cullen, Illinois State University; Douglas 
Clements and Julie Sarama, both of University of Denver. Abstract. 


Developing and Testing a Model to Support Student Understanding of the Sub-Microscopic Interactions that 
Govern Biological and Chemical Processes — Joseph Krajcik, Michigan State University 


Drawing on ongoing work on specific learning progressions, this project examines how high-school students 
progress in their learning of core ideas in physics. Students will interact with new interdisciplinary instructional 
materials for four units of a semester-long course. These materials, which are being validated, include computer 
simulations and embedded assessments that place the students in the learning progression. Co-PI: Shawn Stevens, 
University of Michigan. Abstract. 


Several major reports forecast a need for assessments and assessment practices that 
provide actionable timely feedback for improved teaching and learning. Formative assessment, 
drawing heavily on learning progressions, is a process for determining what a learner knows in 
relation to a learning goal and identifying the best instructional options for advancing their 
learning one step further toward the goal. In contrast to summative and interim assessments, 
formative assessment provides information that is used “to make decisions about the next steps 
in instruction”, !° and results in some form of timely feedback to the learner. It is an essential 


component of effective STEM instruction. 


'° Wiliam, D. (2011). Embedded formative assessment. Bloomington, IN: Solution Tree Press. 
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“States, districts, and others should design, develop, and implement assessments that give 
students, educators, and other stakeholders timely and actionable feedback about student 
learning to improve achievement and instructional practices.” — National Education Technology 
Plan, 2010 


“Assessment and measurement science in the future will require changes in such directions as: 
..1. The blending or integration of assessment, teaching, and learning.” — To Assess, To Teach, 
To Learn: A Vision for the Future of Assessment, 2013 


“Funding agencies should support the development of curriculum and instructional programs 
that include research-based teaching methods, such as: ... Using formative assessment to: (a) 
make learning goals clear to students; (b) continuously monitor, provide feedback, and respond 
to students’ learning progress; and (c) involve students in self- and peer assessment.” — 
Education for Life and Work: Developing Transferable Knowledge and Skills in the 21" Century, 
2012 


At the heart of DR K-12 formative assessment projects is the desire to improve STEM 
learning by providing teachers and students with usable information and next steps. The projects 
represent a variety of formative assessment approaches and types, as well as research on the 
effects of the formative assessment resources and practices.'' For instance, project products may 
include guidance for teacher practice, assessments embedded in published curricula, and 
technology-based simulations that blend assessment and instruction. Below are examples of DR 
K-12 projects that develop training for teachers on formative assessment strategies, construct 
computer-based learning experiences informed by learning progressions, and research on 
effective formative assessment. 


Formative Assessment in the Mathematics’ Classroom: Engaging Teachers and Students — Fred Gross, Education 
Development Center 


The FACETS project is developing and testing a two-year intensive professional development model to build 
teacher capacity in formative assessment strategies. Middle school math teachers will engage in institutes, 
ongoing professional learning communities, and online resources, while incorporating the practices being learned 
into daily instruction. Implementation challenges in the pilot study will inform refinement of the model, which will 
be accompanied by a facilitator’s guide, cyberlearning resources, and recommendations for instituting a 
professional development program based on the model. Abstract. Website. 


Development of a Cognition-Guided, Formative-Assessment-Intensive, Individualized Computer-Based Dynamic 
Geometry Learning System for Grades 3-8 — Michael Battista, Ohio State University 


The Individualized Dynamic Geometry Instruction system (iDGi) will be a computer-based, individualized, 
interactive learning system that can be used by middle school students independently or by classroom teachers 
with groups of students. With four instructional modules in geometry and measurement, iDGi integrates validated 
learning progressions, formative assessments, adaptive instruction and feedback, and features of research-based 
educational media. Testing of the system includes randomized control trial research, after which the project will 
look toward commercial publishing of iDGi including an iPad version. Abstract. 


'' The CADRE website hosts a sortable inventory of DR K-12 projects focused on formative assessment at 
http://www.cadrek12.org/fa-spotlight. 
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Formative Assessment in Mathematics: Current Status and Guidelines for Future Developments — Jamal Abedi, 
University of California-Davis 


A response to the increased use of formative assessment in California’s mathematics classrooms, researchers aim 
to identify which formative assessments and formative assessment practices are associated with improved math 
achievement. The project uses rigorous research methods to identify guidelines and recommendations for the 
development of improved formative assessments. Products include a test blueprint based on state standards and a 
formative assessment prototype for 8th grade algebra. Co-Pl: Paul Heckman, University of California-Davis. 
Abstract. 


8 Sophisticated Assessment Design and Use 


Major national reports point to the urgent need to explore different approaches to 
assessment and accountability systems, with the Gordon Commission making a case that “the 
assessments that we will need in the future do not yet exist.”'” They recommend many 
possibilities for the improvement of assessments, not the least of which involve reevaluating and 
clarifying the intended purposes of assessments, which can drive their design and usability. 


However, two prominent themes emerge from report recommendations. First, technology 
offers opportunities for increasingly sophisticated and efficient assessment, regardless of 
assessment purpose or type. For instance, assessment technologies can be used to save time in 
administration, scoring, and interpretation; provide information that is more accurate and easier 
to use; improve assessment accessibility for a variety of student populations; and enable new 
modes of assessment delivery. Assessments can be incorporated into technology-based 
instructional platforms, such as simulations and intelligent tutoring, or draw on technological 
affordances to produce individualized just-in-time information that a teacher can use for 
instruction. A second prominent theme in the reports is the need for assessments that fit into 
balanced assessment systems that effectively serve both accountability and instructional 
purposes. For instance, there is an interest in developing classroom, benchmark, and large-scale 
assessments that work together, as well as an interest in assessments that can serve multiple 
purposes more effectively than they are served now. 


“Use of assessments: ...Researchers should explore a) how new forms of assessment can be 
made both accessible to teachers and practical for use in the classroom, b) how assessments can 
be made efficient for use in large-scale testing contexts, c) how assessments can be designed so 
that all students have equal opportunities to demonstrate their competencies, d) how 
information from classroom-level assessments and large-scale assessments can be combined 
reliably for use in addressing educational problems; and, e) how various new forms of 
assessment affect student learning, teacher practice, and educational decision making.” — A 
Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas, 2012 


'2 Gordon Commission on the Future of Assessment in Education. (2013). To assess, to teach, to learn: A vision 
for the future of assessment, p. 173. 


© 2013 CADRE 17 


“Build the capacity of educators, education institutions, and developers to use technology to 
improve assessment materials and processes for both formative and summative uses.” — 
National Education Technology Plan, 2010 


“Digital technologies hold great promise for helping to bring about many of the changes in 
assessment that the Commission believes are necessary. Technologies available today and 
innovations on the immediate horizon can be used to access information, create simulations 
and scenarios, allow students to engage in learning games and other activities, and enable 
collaboration among students.” — To Assess, To Teach, To Learn: A Vision for the Future of 
Assessment, 2013 


A 2012 CADRE analysis of the DR K-12 project portfolio found that 100 projects (28 
percent of the portfolio) address assessment.” Many projects in the portfolio have goals of 
developing and testing new assessment methods and technologies, as well as developing 
knowledge intended to inform the field on assessment development and decisions about 
assessment policy and investment. Many of the projects include research to evaluate the impact 
and quality of assessments, including those in the field currently. 


SimScientists Assessments: Physical Science Links — Edys Quellmalz, WestEd 


SimScientists is an interactive, online, simulation-based science assessment system designed to assess students’ 
learning of classroom-aligned content. This project is focused on developing a physical science strand of 
SimScientists with four suites of simulations that can be integrated with existing large-scale testing systems. 
Middle school students engage in simulated scientific activities, such as by watching an animation, making and 
testing their predictions, collecting data, and explaining their findings. The system coaches the student and 
provides reports to teachers, who can adjust instruction. The project will also produce a policy brief with 
recommendation for integrating these types of assessments into district and state systems. Co-Pls: Mark Loveland, 
Daniel Brenner, Barbara Buckley, and Mark Silberglitt, all of WestEd. Abstract. Website. 


CLASS: Continuous Learning and Automated Scoring in Science — Marcia Linn, University of California-Berkeley 


CLASS investigates how science assessments that measure complex reasoning may benefit from automated scoring 
of written-response test questions. The project incorporates automated scoring with five types of science items to 
investigate the impact of various types of feedback on student learning. Whereas scoring written responses 
traditionally was time-consuming and expensive, CLASS aims to take advantage of emerging technology to provide 
efficient real-time feedback to students and teachers. In addition to cutting edge findings for the field, the project 
will produce 21 new standards-based science units, each with four or more automatically scored written response 
items. Co-PI: Ou Liu, University of California-Berkeley. Abstract. Website. 


'S CADRE (2013), p. 11. 
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Evaluation of the Cognitive, Psychometric, and Instructional Affordances of Curriculum-Embedded Assessments: 
A Comprehensive Validity-Based Approach — James Pellegrino, University of Illinois-Chicago 


This research project evaluates the multiple assessment components embedded in two NSF-supported 
mathematics curricula, Everyday Mathematics and Math Trailblazers. Among other objectives, it seeks to 
determine which assessment types are most informative for instructional decisions and for predicting performance 
on external summative assessments, and it will recommend ways to improve assessment activities, scoring 
approaches, and teacher professional development. Project findings are expected to contribute to the field’s 
understanding of what constitutes a quality assessment in elementary mathematics, while also providing an 
innovative model for further research on assessments. Co-Pls: Jim Minstrell, FACET Innovations; Kimberly Gomez, 
University of California Los Angeles; Susan Goldman and Louis DiBello, both of University of Illinois at Chicago. 
Abstract. Website. 


Application of Evidence-Centered Design to State Large-Scale Science Assessment — Geneva Haertel, SRI 
International 


In response to limitations in traditional test development processes, this project is developing a set of effective and 
affordable test development processes that produce interactive standards-based science tasks for incorporation 
into existing statewide testing systems. The project aims to determine ways to enhance the quality of large-scale, 
technology-based assessments and to evaluate these procedures through their use in state-level test 
development. Focusing on middle school science assessment, anticipated products include training materials for 
test developers, exemplar design patterns and science tasks, and resources that allow for efficient item 
development. Co-P/: Robert Mislevy, SRI International. Abstract. Website. 


9 Transferable 21st Century Skills 


There is substantial interest in developing student skills that are transferable across 
content and will contribute to student success in 21“ Century colleges, workplaces, and 
communities. Here referred to as “transferable 21* Century skills,” !* these skills are 
conceptualized under many competing frameworks, yet typically they include competencies 
related to critical thinking, problem solving, communication, collaboration, creativity, 
persistence, leadership, and self-discipline. Several major national reports anticipate that the need 
for these skills will persist and grow in coming years. The reports call for an increased, sustained 
focus on developing assessments and instructional materials that promote the development of 
these skills and prepare students for their future personal and professional success. 


“Foundations and federal agencies should support research to more clearly define and develop 
assessments of 21* century competencies [e.g., critical thinking, communication, flexibility]. In 
particular, they should provide sustained support for the development of valid, reliable, and fair 
assessments of intrapersonal and interpersonal competencies, initially for research purposes, and 
later for formative assessment.” — Education for Life and Work: Developing Transferable Knowledge 
and Skills in the 21 Century, 2012 


'* There are a variety of theoretical and practical models for skill sets described here as “21* Century skills,” 
including those that use the terms “soft skills,” “non-cognitive skills,” “non-academic cognitive skills,” “deeper 
learning,” “social and emotional learning,” “higher order thinking,” and “next generation learning”. 


99 66 ” 
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“Assessment and measurement science in the future will require changes in such directions as...the 
assessment of new skills such as collaborative problem solving and various ‘non-cognitive’ or ‘soft 
skills.”” — To Assess, To Teach, To Learn: A Vision for the Future of Assessment, 2013 


“Research and development funding should be increased for studying the non-cognitive aspects of 
21st-century skills, namely, interpersonal skills (such as such as communication, collaboration, and 
leadership) and intrapersonal skills (such as persistence and self-regulation).” — Expanding Evidence 
Approaches for Learning in a Digital World, 2013 


STEM instruction lends itself to the promotion and development of 21* century skills, 
allowing practice in skills across integrated discipline areas or as a part of project-based learning. 
For example, scientific inquiry and experiments often include collaborative teamwork, predictive 
and analytic thinking, reflection, and communication about outcomes. These skills can be used 
later with other activities and content, and they are skills that are needed in a variety of 
occupations. Many DR K-12 projects incorporate opportunities for students to develop 
transferable 21* century skills through innovative STEM activities. 


Science Literacy through Science Journalism — Alan Newman, University of Missouri-St. Louis 


Science literacy is crucial for engagement in the 21st century economy. Science journalism provides a means to 
deepen students’ science content knowledge and their expository writing skills simultaneously. A model of 
instruction focused on helping students to identify personally and publicly relevant questions, conduct background 
research, synthesize information, fact-check, and write coherently about evidence-based arguments is developed. 
Researchers evaluate the impact of this model on students’ understanding and production of contemporary 
science print and video journalism. Co-Pls: E. Wendy Saul, University of Missouri St. Louis; Cathy Farrar, Normandy 
High School. Abstract. Website. 


Workshop on Assessment of 21st Century Skills — Stuart Elliott, National Academy of Sciences 


This was the third in a series of two-day workshops convened by the National Research Council that explored 21° 
century skills in relation to employer needs, science education reform, and assessment. Convening educational 
measurement specialists, employment analysts, cognitive psychologists, and practitioners in schools, this 
workshop investigated existing assessments, evaluated their usefulness, and discussed current needs for assessing 
Oa century skills. The resulting report presents issues and measurement considerations for assessing oh century 
skills, as well as related policy implications. Abstract. Website. 


EcoMobile: Blended Real and Virtual Immersive Experiences for Learning Complex Causality and Ecosystems 
Science — Christopher Dede, Harvard University 


EcoMobile immerses middle school students into virtual and real ecosystems to create more engaging and 
effective science instruction. Employing smartphones, tablets, and other mobile devices, students investigate 
complex environmental problems through a four-week inquiry-based life science curriculum. The mobile devices 
allow students to access Internet information, collect and share data with one another, and track their location. 
This approach to learning capitalizes on the existing knowledge of technology and social media that many middle 
school students have to enhance their self-directed inquiry, engagement in science learning, communication of 
findings, and content knowledge. Co-P/: Tina Grotzer, Harvard University. Abstract. Website. 
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Agency in Sustained Problem-Based Inquiry: Learning Science Through and As Innovation — John Bransford, 
University of Washington 


This project develops and tests separate biological science units for second and fifth graders that include elements 
of both socio-cognitive and socio-cultural approaches to teaching and learning. Researchers hypothesize that the 
combination of informal and formal learning environments and the incorporation of students’ culturally based 
knowledge and inquiry-based learning will increase student engagement and achievement in science. Additional 
outcomes tracked are students’ ability to work together, their concepts of science and science inquiry, and the 
impact on classroom community. Co-Pls: Carrie Tzou, University of Washington Bothell; Angela DiLoreto, Bellevue 
School District; Philip Bell, Nancy Vye, and Daniel Gallagher, all of University of Washington. Abstract. 


Laboratory for the Study of Extra Solar Planets: Fostering Data Literacy — Roy Gould, Smithsonian Institution 
Astrophysical Observatory 


High school physics students use online telescopes and seven mini-lab lessons to learn about the detection of 
exoplanets. Through this approach, students learn to collect and analyze data, reason from models, and develop 
conclusions from multiple sources of information. The outcomes of this project are a widely available online 
laboratory that engages students in a new area of inquiry-based astronomy learning, as well as a broadened field 
of knowledge about how students learn to understand and interpret data. Co-Pis: Mary Dussault and Susan 
Sunbury, both of the Harvard-Smithsonian Center for Astrophysics. Abstract. 


Through its DR K-12 program, the NSF supports the development of innovative STEM 
education resources, models, and tools for implementation in classrooms, as well as in online 
environments and informal education settings. DR K-12 projects deliver products on the front- 
edge of the field, providing reference points and examples for other developers. The program 
portfolio also includes rigorous research on impact and implementation that produces new and 
valuable knowledge, not just for the assessment of the DR K-12 projects themselves but for use 
by the broader U.S. R&D community. 


In recent years, nationally prominent committees, comprising experts from a variety of 
fields, have produced recommendations for the improvement of PreK-12 STEM education in the 
U.S.. Their reports each propose priorities and next steps, and viewed collectively, the 
recommendations point to priority areas that reflect shared concerns and hopes. The priorities 
that emerge across multiple reports provide guidance for strategic thinking on current and future 
R&D efforts. 


DR K-12 projects provide illustrative examples of R&D work that is aligned with 
national priorities for STEM education that have been expressed in major reports. To the extent 
they are successful, projects in the DR K-12 program contribute to advances in these priority 
areas and improve student achievement and interest in the STEM fields. 
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Appendix: Major National Reports on Science, Technology, 


Engineering and Mathematics (STEM) Education 


A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas 
Published in 2012 by the National Research Council and the Committee on a Conceptual 
Framework for the New K-12 Science Education Standards. Available at 
http://www.nap.edu/catalog.php?record_id=13165. 


Coordinating Federal Science, Technology, Engineering, and Mathematics (STEM) Education 
Investments: Progress Report 
Published in 2012 by the National Science and Technology Council and the Committee on 
STEM Education. Available at 
http://www. whitehouse. gov/sites/default/files/microsites/ostp/nstc_federal_stem_education 


coordination_report.pdf. 


Education for Life and Work: Developing Transferable Knowledge and Skills in the 21° Century 
Published in 2012 by the National Research Council and the Committee on Defining 
Deeper Learning and 21“ Century Skills. Edited by Pellegrino, J.W., & Hilton, M.L. 
Available at http://www.nap.edu/catalog.php?record_id=13398. 


Expanding Evidence Approaches for Learning in a Digital World 
Published in 2013 by the U.S. Department of Education’s Office of Educational 
Technology. Available at http://www.ed.gov/edblogs/technology/evidence-framework 


Expanding Underrepresented Minority Participation: America’s Science and Technology Talent 
at the Crossroads 
Published in 2011 by the National Academy of Sciences; National Academy of 
Engineering; Institute of Medicine; Committee on Underrepresented Groups and the 
Expansion of the Science and Engineering Workforce Pipeline; Committee on Science, 
Engineering, and Public Policy. Available at 
http://www.nap.edu/catalog.php?record_id=12984. 


Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing? 
Published in 2012 by the National Research Council and the Committee on the 
Evaluation Framework for Successful K-12 STEM Education, Board on Science 
Education, Board on Testing and Assessment, Division of Behavioral and Social 
Sciences and Education. Available at http://www.nap.edu/catalog.php?record_id=13509. 


National Education Technology Plan 2010 (Transforming American Education: Learning 
Powered by Technology) 
Published in 2010 by the U.S. Department of Education’s Office of Educational 
Technology and the National Education Technology Plan Technical Working Group. 
Available at http://www.ed.gov/technology/netp-2010 
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Prepare and Inspire: K-12 Education in Science, Technology, Engineering, and Math (STEM) 
for America’s Future 
Published in 2010 by the President’s Council of Advisors on Science and Technology 
(PCAST) and the Executive Office of the President. Available from 
http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stemed-report.pdf 
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